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The research results I will present in this paper— and try to 
interpret— were drawn from a series of studies carried out over the 
past eight or nine years. The primary focus of these investigations 
has not been on the processes and phenomena of interpersonal 
perception as such. Rather, the emphasis has been on the development 
of alternative methods to improve the assessment of personality 
attributes within the normal range of human variation. Indeed, the 
earlier studies in this series took the results of peer-rating 
assessments as criteria against which other procedures, stimulus 
materials, and scoring methods were to be "validated." 

Previous findings by Fiske (1949), Borgatta (1960, 1964), 

Tupes (1957), and Tupes & Christal (1958, 1961) had indicated that 
a stable factorial structure existed for the given set of peer-rating 
scaUes across diverse conditions and populations and that scores 
derived from these scales predicted tJ military performance criteria 
in the Air Force. Subsequent studies have tended to corrcjorate 
and extend these findings both as regards the stability of the 
factorial structure and the predictive power of the scales against 
a variety of performance criteria in various contexts. The tentative 
names assigned to the five factors in this structure are: I. Extro- 

version or Surgency^ XI. Agreeableness; III. Conscientiousness; 

IV. Emotional Stability; and V. Culture. 

The initial studies in the present series attempted two things; 
first, to replicate and extend to yet another population the previous 
findings regarding the factorial structure of the peer-rating scales 
and second, to develop and validate self-report measures of these 
dimensions against the peer-ratings considered as (intermediate) 
criteria. Accounts of these efforts have been published previously 
(Norman, 1963a and Norman, 1963c) as have the results of some additional 
and unsuccessful efforts to employ maximal performance tests and 
preference responses to content irrelevant stimuli to assess these 
factors (Norman, 1963b). In brief, moderate convergent and discriminant 
validities were achieved for a five-factor structure by use of content 
relevant stimuli presented in forced-choice formats. In addition, 
specially designed scales for detecting faking and a novel scoring 
procedure for the content factors yielded substantial control over 
tendencies to distort self-reports on these Instruments. 

While a moderate degree of success was achieved by these efforts , 
the convergent validities were neither so high as one might wish for 
applied purposes nor so large as the respective reliabilities would 
theoretically permit. Clearly there were systematic (non-random) 
sources of variance present in the various sets of measure variables 

iThe author is grateful to Eva F. Bradford for her assistance in 
the collection and analysis of much of the data included in this 
report. The studies reported were supported in part by Research 
Grant MH-07195 from the National Institute of Mental Health, United 
States Public Service. 
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that were unrelated to the personality constructs these measures 
were intended to assess. Moreover, some of these extraneous deter- 
minants of score variance could easily be identified as purely 
methodological in character , 

For example, the peer-ratings were obtained by forcing distri- 
butions of nominations by each rater. A consequence of this forced 
nomination procedure was that all groups of ratees necessarily 
obtained identical mean scores on all rating scales. Since the self- 
report measures were not similarly constrained by the procedures used 
to collect and score them, a systematic difference in method variance 
components existed and thus served to attenuate validities. 

In like manner the (quasi-) counterbalanced, forced-choice format 
employed for the self-report instruments resulted in mean profiles 
for all respondents that had (nearly) constant "elevations” or mean 
levels across scales. No such restraint was present in the peer- 
rating measures and this presumably limited even further the 
convergent validities attainable. 

Efforts by Norman and Harshbarger (1965) to identify and estimate 
the magnitudes of the various methodological and substantive 
constituents of these measure variables and to devise adjustment 
formulas to bring them into closer mutual alignment met with only 
modest success. For four of the five personality factors, varying 
amounts of Increase in the correlations were obtained, averaging 
about .10 points, while for the fifth factor little if any increase 
resulted from any of the adjustments. 

Thus, while it was possible to identify a number of format and 
procedural components in each of the respective sets of assessments, 
adjustments or corrections devised to remove these methodological 
artifacts were only partially successful. There remained substantial 
portions of reliable variance in each set of assessments still 
unaligned with that in the other set. Accordingly our attention 
turned to other potential sources of this residual variation and 
we were led to consider other means of identifying and estimating 
the magnitudes of such components. 

Since the self-report inventories we had been using yielded 
scores that were both (quasi-) ipsatized and subject to various 
amounts of multiple keying of single response alternatives we 
constructed two new sets of self-report measures that would be free 
of these artifacts. The first was a set of statistically and conceptually 
homogeneous keys for an inventory composed of ordini.ry self-report 
items. The second set of variables was constructed by the simple ex- 
pedient of attaching to each of the bipolar peer-rating forms a 
graphic scale on which the respondent could register a self-rating 
as well as his estimate of where his peers would place him on this 

scale. 

These measures, together with the peer— rating scales, were 
administered to several samples of subjects. Small pilot samples of 
firemen and fraternity seniors, a larger group of fraternity men, a 



( 
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sample of Peace Corps Trainees and a large group of university 
student volunteers were tested with various parts of the battery. 

The major analvses were carried out on the group of 169 fraternity 
men who responded on all three sets of measures. Only the scales 
for the first four factors were used in these analyses because of 
the limitations of the self-report item pool relative to the fifth 
factor. 

The analysis of these data proceeded in two related directions. 

The more direct approach was based on the rationale that if the 
subsets of variables within each of the three types did in fact tap 
the dimensions they had been designed to measure, then a joint 
analysis of all the variables should yield a factor pattern in 
which four content factors should emerge, each with three subsets 
of salients (corresponding to the variables of each type that had 
been designed to tap that factor). If, in addition, all variables 
of a given type differed from those of other types in some systematic 
manner, then one might expect one or more methods factors also to 
appear in the pattern. Table 1 presents a schematic representation 
of such a possible factor pattern in which there are four P^re, 
heteromethod, content factors and a pair of bipolar (differential) 
method factors. The method factors portrayed contrast peer vs. 
self sources of data and stimulus presentation and/or response 
format differences within those measures based on the two types 
of self-reports. Other kinds and combinations of method factors 
would also be possible, of course, and those presented in Table 1 
are meant only to be illustrative. 

There exists a major technical reason for questioning the 
feasibility of this approach, however. Note that the hypothesized 
factor pattern given in Table 1 does not conform very well to the 
simple structure prototype. In particular, note that apry. ^ ® 
is presumed to have a complexity greater than one and that the hyper- 
p)ane counts for the method factors would presumably be very low- 
in fact, approaching zero in some cases. Such features of a data 
set are anathema to rotational methods based on simple structure 
criteria. Since most of the commonly used analytic methods (including 
Varimax and Biquart imin) are based on such criteria, efforts to rotate 
data matrices with structures like that given in Table 1 by means 
of such methods might well fail to attain such a solution. 

An attempt was made, however, to determine whether convergent 
loading patterns could be obtained directly by an application of 
these standard rotational criteria to the principal axes of each 
of the monomethod sets of variables as well as those for each hetero- 
method combination of the variables. The general findings of these 
analyses were that (a) Varimax and Biquartimin solutions for single 
(monomethod) sets of variables displayed the expected dimensionali y 
and good approximations to simple structure patterns in all three 
cases but, (b) neither Varimax nor Biquartimin solutions for joint 
(heteromethod) combinations of variables gave adequate approximations 
to the type of pattern presented in Table 1. Only rudimentary 
indications of the convergent pattern were detectable in some columns 

of each of these solutions. 
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Table 2 presents the normalized Varimax solutions for the three 
sets of variables analyzed separately* Table 3 presents the 
normalized Varimax solution for all three sets of variables analyzed 
jointly. The similarity of the results in Table 3 with the projected 
pattern presented in Table 1 is not very great— even if only those 
columns which most closely approximate the content factors of the 
hypothetical pattern are considered. Clearly a rotational method 
based on simple structure criteria (i.e,, Varimax) was badly suited 
to the analysis of these non-slmple structure data* Bi<iuartimin 
solutions which were obtained for these same data were hardly any 
better. 

Fortunately, however, there does exist an alternative rotational 
method that does permit the investigator to test his prior hypotheses 
concerning the structure of complex sets of variables such as 
these. It carries the unfortunate and, as I hope to show, somewhat 
misleading name of ’‘Procrustes.” In this approach the investigator 
is allowed (and required) to specify fully his hypothesis in the form 
of a numerical factor pattern. Tne principal axis solution (or 
some other derived configuration) is then rotated into the closest 
possible approximation to this pattern that the intrinsic structure 
of the data permits. The solution is thus a ’’fitted" one. Accordingly 
the danger exists (as the name "Procrustes" suggests) that the data 
may be "cut" or "stretched" to fit the specified "Procrustean bed," 
willy-nilly. Horn (1965) has in fact shown that such a possibility 
exists with "data" that are without any intrinsic struc^re— i.e, , 
entirely random. However, as the right sides of Tables 4, 5, and 
6 show, there are clearly limits to the extent that in t r i^s leal l x 
structured data such as those presently under consideration can be 
wrenched to fit a specified pattern. These "worst" solutions were 
obtained by specifying a pattern which was chosen to be as thoroughly 
contrary as possible to that hypothesized to exist in these sets 
of variables. Clearly these data could not be made to fit, with 
any detectable comfort, so harsh a bed! 

The solutions given on the left of Tables 4, 5, and 6 are those 
obtained by specifying a "Best" a priori pattern based on the 
design of these batteries. These oblique solutions are somewhat of 
an Improvement over the corresponding varimax solutions presented 
in Table 2 although the correlations among the primaries are not 
vary high in most instances. 

Thus it was found that by specifying a target pattern 
to be fitted in each case by a Procrustes procedure very close fits 

to the a priori specifications could be obtained. What- -is more, 
efforts to fit these data to alternative specifications that were 
markedly disparate from those based on the psychometric design of 
the variables produced extremely poor fits. Thus, the sometimes 
expressed opinion that a Procrustes procedure is capable of wrenching 
just any data matrix to conform to a given specification or target 
pattern is clearly not a valid one, at least for data as highly 
structured as these. 



Note— Loadings <1*^1 and decimal points for loadings and communalities omitted. 
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Table 2 

Normal Varimax Solutions Based on 169 Fraternity Men 



I 



-7- 

Table 3 

16 Peer Rating Plus 16 Self Rating Plus 24 Questionnaire Variables 
169 Fraternity Men - Normal Varimax Solution 



Variable Variable Factors 2 

Set No^ I II? Ill PR? SR? QR? h 





1 


78 










72 




2 


54 










39 




3 


64 




-36 


31 




71 




4 


69 




-40 






68 




5 




33 




72 




64 




6 








76 




68 




7 




34 




72 




71 


Peer 


8 








74 




69 


Ratings 


9 






72 






66 




10 






72 


46 




77 




11 


-32 




60 


40 




70 




12 






69 


42 




72 




13 








73 




59 




14 








70 




56 




15 








73 




58 




16 








60 




43 




1 


68 










48 




2 


57 








• 


36 




3 


50 








34 


42 




4 


48 










28 




5 




40 








28 




6 










46 


35 




7 




31 






30 


29 




8 


42 








33 


37 


Self 














9 






61 






43 


Ratings 


10 






72 






61 




11 






56 






40 




12 






58 




33 


47 




13 










53 


36 




14 










67 


61 




15 










33 


33 




16 












10 



(Table continued on next page) 
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Set 



Table 3 (continued) 

16 Peer Rating Plus 16 Self Rating Plus 24 Questionnaire Variables 
169 Fraternity Men - Normal Varimax Solution 

Factors 



Questionnaire 



Variable 

No^i^ 



1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 



Factor Contributions 



40 

41 
51 
34 
46 

42 



5.03 



II? 



30 

32 



38 

77 

66 

63 

63 

62 



III 



PR? 



SR? QRI 



38 

30 



64 

61 

63 

55 

44 

67 



34 



58 

51 

49 



39 

30 

43 

31 

74 

71 

79 

76 

68 

61 



Note-Loadings < l*.30l and decdmal points £ot loadings and 
communalities omittede 



38 

38 

50 
57 

51 
44 

27 

61 

48 

53 

48 

46 

57 

40 

52 

32 

40 

56 

59 

56 

66 

51 

39 



3.84 6.69 5.16 2.23 5.11 28.06 
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Table 4 

16 Peer Rating Variables - 169 Fraternity Men - Procrustes Solutions 



"Best” A priori Solution “Worst” A priori Solution 



Wo. and 
Name 


Variable 

No. 


I 


II 


III 


IV 


A 


B 


C 


D 




1 




81* 








47* 


32 




43 


I 


2 




71* 








45 


66* 


59 


83 


Extroversion- 


3 




64* 








55 


72 


46* 


40 


Surgency 


4 




68* 








49 


44 


82 


34* 




3 






70* 






50 


40 




42* 


II 


6 






54* 








35 


38* 


60 


Agreeableness 


7 






67* 








(26>* 








8 






56* 






(20)* 




60 


32 




9 








67* 




39 


(22)* 


44 


41 


III 


10 








76* 




(07)* 


30 


53 


58 


Dependability 


11 






32 


60* 




57 


68 


45 


(29)* 




12 








80* 






31 


64* 


49 




13 










42* 






(18)* 


31 


IV 


14 










70* 




52 


46 


81* 


Emotional 


15 










68* 


(29)* 


56 




38 


Stability 


16 










51* 




(23)* 


75 


30 






I 








A 










Primary 




II 


08 






B 


-86 








^.actor 

Correlations 




III 


-34 


25 




C 


-51 


29 










IV 


17 


57 


14 


D 


63 


-80 


-40 





Note-* - a specification of .99 for this loading. All other specifications set - .00 

<l*.30| not included except for those in starred positions in which 

case they are put in ( ) . Decimals omitted. 
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Table 5 



16 Self Rating 


Variables - 


169 Fraternity Men - Procrustes Solutions 


Factor 
No. and 
Name 




"Best" 


' A priori Solution " 


'Worst*' 


A priori Solution 


Variable 

No. 


I 


II III IV 


A 


B 


C 


D 




1 


65* 




49* 






34 


I 


2 


55* 




48 


(20)* 






Extroversion- 

Surgency 


3 


47* 




33 




37* 


49 




4 


53* 




44 






(23)* 




5 




49* 








49* 


II 


6 




32* 35 






46* 


47 


Agreeableness 


7 




56* 




49* 




47 




8 




48* 


48* 


58 




61 




9 




59* 




30* 






III 

Dependability 


10 




75* 


56* 


58 








11 




43* 




50 




(18)* 




12 




61* 


54 


46 


36* 






13 




56* 






64* 


37 


IV 


14 




61* 






68 


43* 


Emotional 

Stability 


X 5 




45* 


(28)* 




50 






16 




(17)* 




(16)* 








I 




A 










Primary 

Factor 


II 


14 


B 


-63 








Correlations 


III 


06 


21 C 


-52 


65 








IV 


31 


29 06 D 


23 


-66 


-68 





Note—* * a specification of .99 for this loading. All other specifications set * .00 
loadings <|*.30| not Included except for those in starred positions in which 
case they are put in ( ). Decimals omitted. 
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Table 6 

24 Questionnaire Response Variables - 169 Fraternity Men - Procrustes Solutions 

“Best” A priori Solution “Worst” A p riori Solution 

I II III IV A B C D 



Factor 
Mo. and 



Variable 



I 

Ext rovers ion- 
Surgency 


1 

2 

3 

4 

5 

6 


76* 

50* 

61 * 

60* 

57* 

38* 


30 




40 


76* 

38 

45 

59 

53* 


(27)* 


64 

44 

53* 

69 

71 

48* 


34* 

31 




7 


46 


42* 






65* 




56 


47 


II 


8 




69* 






36 






68* 


Agreeableness 


9 




68* 






43 




32* 


69 




10 




68* 








(07)* 




64 




11 




64* 






31* 






66 




12 




64* 






38 




33 


71* 




13 






78* 






77* 


37 


36 


III 


14 






70* 






62 




(21)* 


Dependability 


15 






75* 




(12)* 


71 


31 






16 






56* 






50 


(24)* 






17 






60* 


33 




72* 


47 


42 




18 






56* 


31 




67 


36 


49* 




19 


30 






30* 


34 


48* 


60 


35 


IV 


20 








44* 




54 


55* 


51 


Emotional 


21 








68* 


(22)* 


1 


61 


47 


Stability 


22 








48* 


30 


30 


59 


46* 




23 






51 


51* 




74* 


56 


58 




24 








40* 




30 


50* 


39 



Primary 


II 


-05 


pactor 


III 


28 


Correlations 


IV 


24 



Note--* - a specification of .99 for this loading. All other specif iwtlons set - .00, 

<|i.30l not included except for those In starred positions in which 

case they are put in ( ) . Decimals omitted. 
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However, our major interest was in the extent to which the full 
heteromethod battery possessed a 

in Table 1. The Varimax solution presented in Table 3 gave little 
assurance that such was the case— but perhaps for reasons of 
inappropriateness already mentioned. Table 7 presents a Procrus 
solution for these data based on a specification like that in Table 
1. A very clear pattern of heteromethod convergence is reflected 
in the first four columns of these results. A pair of ^at method 
fetors contrasting, respectively, peer rating with self-reports and 
self-ratings with questionnaire responses within the self-report 
domain rotmd out the picture. 

Whereas the analyses described above have attempted to display 
the amount of heteromethod convergence in these data directly in 
terms of the coefficients of the factor patterns, the second approach 
to the analysis utlUped a more indirect three-stage 
the variables of each type vere separately factor analyzed ^d 
rotated— in this case to simple structure solutions (cf, ’Cables 
Tables 4 5, and 6). Next, factor score estimates were computed 

ffr eLh s^ject on each factor of each of the separate analyses, 
tod finely the factor score estimates from all analyses were inter- 
^rrfiaSd! H the several sets of variables each yielded measures 
ofthe same factors then the correlations obtained in this manner 
stould display what Campbell and Fiske (1958) have termed a convergent 
and discriminant validity pattern. 

For purposes of this analysis, each set of variables actually 
vielded three sets of factor score estimates. The first two sets 
were multiple regression estimates based, respectively, on a normalized 
Varimax solution and a Procrustes rotation of the princip^ ^es. 

The third set of estimates was based on a simple unit-weighting 
of the a priori salients for each factor presumed to be present in 

each set of data. 

It is clear from the separate factor tables already presented 
that the structures of the several sets of measures are highly 
and that, except for a few exceptions, those variables 

presumed to tap each of the factors do so. It is *2®* 

even the orthogonal (normalized Varimax) solutions given in Table 2 
approximate a simple structure form rather closely, and the oblique 
(Procrustes) patterns in Tables A, 5, and 6 do so to an even higher 

degree. 

But the proof of convergence lies not in comparisons amtog the 
separateirobWlned factor patterns but rather in the correlations 
ao^ng the sets of factor score estimates. Table 8 
multitrait-multlmethod matrices based on the three factor 
e^imation procedures utilized. It also presents the correlatitos 
b«w^ estimation procedures for each trait-by-data-source combination. 

The first thing to note about the results in Table 8 is that 
the methods of deriving factor score estimates (i.e., ° 

estimates from Varimax and Procrustes solutions and unU-weigh , 
a priori salients) all yield very highly similar results. This is 
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Table 7 

16 Peer Rating, 16 Self Rating, and 24 Questionnaire Variables 
169 Fraternity Men - Procrustes Solution 



Variable 



Variable 

Mo. 



Factors 



III 





1 


74* 








( 09)* 






2 


52* 








( 22)* 






3 


54* 








( 23)* 






4 


59* 








(-03)* 






5 




50* 






37* 






A 




43* 






41* 




Peer 


7 




47* 






40* 




Ratings 


9 

8 




48* 






( 29)* 






9 






67* 




(-15)* 






10 






71* 




( 17)* 






11 






57* 




( 20)* 






12 






67* 




( 24)* 






13 




34 




33* 


34* 






14 








45* 


36* 






15 








36* 


48* 






16 








33* 


35* 






1 


63* 








(-16) § 


( 08)oe 




X 

2 


55* 








(-14) § 


( 01). 




a 


37* 








(-24) § 


( 28). 




4 


42* 








(-26)§ 


( 13). 








48* 






(-22) § 


( 11)« 




(i 




42* 






(-21) § 


( 27). 


Self 


o 

7 




41* 






(rl9)§ 


( 25). 


Ratings 


9 

8 


30 


(28)* 






(-20) § 


35. 




Q 






58* 




(-17) § 


(-08). 




1 n 






74* 




-33§ 


( 27). 




XV 

11 






54* 




(-18)5 


( 06). 




Xx 

12 






60* 




-30§ 


( 21). 




1 % 








40* 


-345 


( 24). 




Li 








63* 


-305 


30. 




14 
1 ^ 








41* 


(-04>§ 


( 23). 




13 

16 








(16)* 


(-16) § 


( 18). 



(Table continued on next page) 



Table 7 (continued) 



16 Peer Rating, 16 Self Rating, and 24 Questionnaire Variables 
169 Fraternity Men - Procrustes Solution 



Variable 
Set 



Variable 
No. 



II 



III 



Factors 
IV 









1 

2 

3 

4 

5 

6 



44* 

41* 

59* 

41* 

36* 

49* 



-36§ (-23)§ 
(-01) § (-00) § 
( 01) § (-26) § 
-40§ -365 
(-14)5 -345 
(-15)5 -385 



Quest lonnalre 



7 

8 
9 

10 

11 

12 



38* 

70* 

63* 

61* 

59* 

57* 



-325 (-08)5 
(-22)5 (-27)5 
(-19)5 (-21)5 
(-23)5 (-11) § 
(-25)5 (-18)5 
(-29)5 (-20)5 



13 

14 

15 

16 

17 

18 



61* 


(-22)8 


(-17)5 


61* 


(-19)5 


( 16)5 


61* 


(-29) 5 


(-03) 5 


54* 


(-09) 5 


( 01)5 


39* 


(-19) 5 


(-28) 5 


63* 


(-27)1 


(-16) 5 





Hote— * - a specification of .70 for this loading; « = a specification of .50 for 
this loading; § « a specification of -.50 for this loading. All 
s5eciflcSs set - !oO. Loadings < l*.30l not included except fr those 
in non-zero_specification positions in which -se they are put in O- 



Decimals omitted. tL contrasts Peer vs. Self sources of data. 
Rating vs. Questionnaire formats within the Self source. 



M 2 contrasts 



f 
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Table 8 

Correlations Among Factor Score Estimates from Three Data Sources 

and Three Methods of Derivation 
N * 169 Fraternity Males 



P 

r 

o 

c 

r 

u 

s 

t 

e 

s 



U 

h 

i 

t 



e 

i 

g 

h 

t 



PR 



Varimax 



Peer Ratings 
T IT III IV 



Self Ratings 



Questionnaire Scales 
T II 111 IV 






I 


[56] 


-15 


-12 


SR 


II 


05 


[29] 


06 




III 


-07 


-05 


[63] 




IV 


11 


06 


-04 




I 


[39] 


-10 


14 


QR 


II 


-23 


[44] 


01 




III 


-14 


—01 


[55] 




IV 


10 


-04 


11 



02 


(83) 






05 


06 




22 


-00 


09 


(89) 


301 




12 


-01 


03 


[53] 


07 


01 


01 


-Z3 


[34] 


-04 






r061 -01 



00 

-08 



[70] 

08 



(82) 


03 


(89) 






10 


-01 


(^ 


01 


05 


-01 


(W 


[3VL 


07 


02 


04 






I 








57 








II 




94 






30 


60 




III 




21 






30 


IV 








95 






I 


56 






2i 


2§. 






II 




26 






2i 




III 






62 






2i 


IV 








30 






I 


37 






45 


32 






II 




42 






69 




III 






57 






35 


IV 








05 







35 



39 



54 



07 



52 



31 



69 



32 



91 



99 



98 



97 



I 


96 








55 








II 




94 








28 






III 






2i 








60 


29 


IV 








2i 








I 


51 








2i 


92 






II 




31 








96 




III 






62 








89 


IV 








32 








I 


39 








47 








II 




41 








35 


69 




III 






57 








33 


IV 








06 









33 



39 



53 



06 



53 



36 



69 



31 



89 



97 



96 



97 



kmc 
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Table 8 (continued) 

Correlations Among Factor Score Estimates from Three Data Sources 

and Three Methods of Derivation 
N » 169 Fraternity Males 



Procrustes 


Peer Ratines 


Self Ratines 


Questionnaire Scales 


I II III IV 


I 11 III IV 


I II III IV 







II 


V 


PR 


III 


a 




IV 


r 




I 


1 




II 


m 


SR 


III 


a 




IV 


X 




I 




QR 


II 






III 






IV 



P 

r 

o 

c 

r 

u 

s 

t 

e 

s 



PR 



SR 



QR 



I (87)" 

II 09 

III -38 



(77 

28 

64 




-34 17 

(86t--44 
16 (70 



I [57] 


07 


-23 


08 


(80)^ 




06 


II 11 


[28] 


09 


16 


24 


(^ 




III -15 


01 


[59] 


-14 


04 


25 


(W 


IV 21 


16 


-07 


[31] 


39 


35 


-00 


I [31] 


-07 


18 


04 


[45] 


08 


21 


II -18 


[38] 


12 


12 


-05 


:so] 


-01 


III -15 


-04 


[53] 


-05 


-03 


04 


[68] 


IV 04 


08 


16 


[08] 


04 


03 


12 



(793 


18 


(79T^ 




28 


24 


17 


00 


(^ 


-.-48 


38 


04 


47 


-01 


(W* 




[33] 


43 


34 


25 


(^ 



U 

n PR 

i 

t 

W SR 

e 

i 

g 

h QR 
t 



I 

II 

III 

IV 

I 

II 

III 

IV 

I 

II 

III 

IV 



99 



52 



34 



99 



34 



39 



99 



60 



55 



98 



31 



08 



56 



97 



49 



:7 



94 



33 



58 



97 



67 



30 



90 



31 



28 



44 



96 



38 



35 



97 



52 



69 



98 



08 



32 



98 



# 
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Table 8 (contlnued-2) 



(joffglations Among Factor Score Estimates from Three Eiata Sources 

and Three Methods of Derivation 
N * 169 Fraternity Males 



Unit Weight 


Peer Ratines 


Self Ratines 


Questionnaire Scales 


I II III IV 


I II III IV 


I II III IV 



V 

a 

r 

1 

m 

a 

X 



PR 



P 

r 

o 

c 

r 

u 

8 

t 

e 

s 



SR 



QR 



U 

n 

1 

t 

W 

e 

1 

g 

h 

t 



I 

II 

III 

IV 

I 

II 

III 

IV 

I 

II 

III 

IV 



PR 


II 

III 




IV 




I 


SR 


II 

III 




IV 


QR 


I 

II 




III 




IV 




I 


PR 


II 

III 



Note~Values along main diagonal (in parentheses) for Varlmax 
and Procrustes sets are multiple Rs of factor score 
estimates against factors. Values in mono-source, 
mono-rotation triangles 



above main diagonal 
for Varimax and Procrustes sets are the correlations 
among the primary factors. 




IV 



08 

-44 

16 



30 

60 



16 





I 


[51] 


-11 


-25 


01 


18 






SR 


II 


09 


[32] 


06 


22 


20 




III 


-26 


-01 


[60] 


-14 


-01 


10 




IV 


12 


12 


05 


[31] 


22 


35 




I 


[31] 


-03 


11 




[49] 


11 


14 


QR 


II 


-20 


[38] 


17 


15 


-01 


[38] 


10 


III 


-19 


-02 


[54] 


-05 


-05 


08 


[69] 




IV 


02 


02 


17 


[07] 


-01 


-00 


17 



21 

13 

13 

[31] 



13 

33 

50 



11 

21 



34 
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reflected most clearly in the underlined diagonals of correlations 
between derivation methods for each of the trait-by-data source 
combinations. These 36 values range from .89 to .99 with an overall 
median of .97. The correlations between the Varimax and the Procrustes 
estimates and those between the Procrustes and the unit-weighted 
estimates are slightly higher than those between the Varimax and the 
unit-weighted scores. Thus it would appear that a choice among 
these procedures for estimating scores of persons on the factors that 
are implicit in these several data sources is not a very crucial 
consideration. 



The more fundamental findings, however, are contained with the 
multitrait-multisource submatrices of Table 8. There one will 
note that the validity diagonals contain vaues which, with one 
common exception in each submatrix, portray a moderately high level 
of convergent and discriminant validity. For example, in the 
Varimax submatrix, only the validity for Factor IV between the 
neer-ratings and questionnaire sources fails to exceed all of the 
relevant heterotrait-monosource and heterotrait-heterosource entries. 
Indeed, the magnitudes of the validities not only stand out from 
the (generally low) heterotrait values but they are in several instances 
quite substantial nunerically as such estimates generally go. While 
there is apparently still some reliable variance in each of these 
sets of measures that is not common across data sources (as witness 
the multiple correlations in parentheses along the main diagonal 
relative to the corresponding validities) there is nonetheless 
an appreciable proportion of common variance across sources, ranging 
to near 50% in a few cases. And finally, as a further reflection 
of the comparatively small proportions of unique variance (specificity 
plus error) in these factor measurements and those based on the 
Procrustes method, note the close correspondence between the monosource, 
factor measurement correlations (just below the main diagonal) 
and the primary factor correlations (presented just above the main 
diagonal) as obtained from the various factor analyses and rotations 
done earlier. The latter reflect only common factor sources of 
variance while the former are affected by all variance components 
present in the measure variables. 

The results based on the Procrustes solutions are a bit more 
difficult to summarize easily because of the obliquities among the 
factors within each monosource analysis. Note, however, that each 
validity can be compared with twelve heterotrait entries. A ratio 
of the number of these comparisons in which the validity entry is 
the larger to the total number (twelve) gives a rough index of the 
exteirt^which discriminant validity obtains. If di«crimd^nant 
validity is absent, this index should be about .5 and it should 
approach unity as discriminant validity increases. With the exception 
of the correlation between the peer rating and the 
measure of Factor IV where this index is .25, the remaining eleven 
validities have discriminant indices that range from . . 

wiih a medi^ forall twelve of .83. Discriminant validity is obviously 

best L^Factor III and poorest for Factor IV. ^ 

the scmewhat lower magnitudes of these validities (relative to 
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based on the Varimax solutions) should be conditioned in terms of 
the multiple correlations along the main diagonal which are also 
somewhat lower for these oblique solutions. 

Finally, the results in the unit-height submatrix correspond 
very closely to those based on the Procrustes solutions and 
accordingly warrant little additional comment. It is, however, 
rather astonishing just how closely the results of this very 
simple method match those obtained by the very arduous and complex 
procedures entailed by the multiple regression method based on the 
Procrustes solutions. Apparently one can do a relatively good job 
of measuring factor scores of persons very simply if he takes the 
trouble to construct adequate measure variables and to constitute 
heterotrait batteries to appropriately map the domain of interest. 

The most glaring failure to obtain evidence of convergent and 
discriminant validity in these analyses occurred for Factor IV, 

Emotional Stability, as measured by the peer ratings and the quest ionnai 
scales, respectively. Actually each of these measurements showed 
a positive (albeit, moderate) relationship to the measurement of 
this factor based on self ratings. But apparently the combined 
effects of source differences together with stimulus presentation 
and response format differences, were, in this one area of content, 
sufficient to wash out content convergjence. 

Whether this result is simply a consequence of poor measure 
variable design or whether the psychological processes that bear 
on assessing this molar attribute are markedly different when the 
object is the self vs. when it is others is not entirely clear. 

More than likely it is some combination of the two. 

It is interesting to note, however, that a factor similar to 
Emotional Stability (or its bipolar reflection) has been identifie 
as a major source of variance in many of the most widely used 
personality inventories (cf. Block’s recent analysis of the 
1965). It is also true that previous efforts to establish conve g 
validity for the scales of such inventories against external rating 
criteria have frequently been highly discouraging. Such resu ts 
have occasionally been interpreted as a generalized 
and limitation of personality measurement as a x^hole. It s 
aging to speculate that these previous failures, including the one 
reported here, may reflect merely on the difficulty of assessing 
one (albeit, an important one) out of many personality factors of 
general interest and importance. 

While each group of factor score estimates (derived from either 
the Varimax or the Procrustes solution) was based on independen 
seL of responses these data were all obtained from the same subjects. 
Thus while the validities presented in Table 8 are not fitte 
in any sense that capitalizes on random error variation there is 
a queLion concerning how widely these functions can be generalized 
to other samples from this or other populations. Tentative xndi- 
catlons based on the two small pilot samples mentioned earlier are 
presented in Tables 9 and 10. Some appreciable drops in convergent 
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validities can be noted; particularly in the case of Factor III 
for the sample of firemen. On the other hand, the convergent 
values for Factor IV are actually higher for both of these groups 
than they were for the original sample on which t e factor score 
weights were derived. Additional samples from more diverse 
subpopulations will be required, of course, before any very 
final evaluation concerning the generalizability of these findings 

can be made. 



The results presented thus far have dealt mainly with the 
relationship between peer-rating assessments and self-reports 
derived either from graphic rating scales or itemized inventory 
responses. As mentioned earlier, however, some of the samples of 
respondents were also asked to estimate where on the gtap ic s*-a es 
their peers would rate them. For two of these samples, a group 
of 73 Peace Corps trainees and a group of 84 previously unacquainted 
university undergraduates, these predicted peer-ratings were also 
separately factor analyzed and five-factor solutions were derived 
by normal varimax rotations. Multiple-regression estimates of 
factor scores were computed for each subject from each of the three 
sets of variables separately and these factor-score estimates 
from each of the three data sources were then intercorrelated 
for each sample. The results are presented in Table 11 (taken from 
a previously published paper; Norman & Goldberg, 1966). 

The correlations for the Peace Corps trainee sample given at the 
top of Table 11 display a pattern of moderate convergent and discriminant 
validity between the peer-ratings and self-ratings similar to that 
previously obtained with other groups of close associates. The 
previously unacquainted subjects in the second sample, however, 
yielded iLer values in the convergent validity diagonal with those 
for Factors II and IV failing even to reach statistical significance. 

But perhaps the more interesting findings are those between 
the predicted peer-ratings and the other two sets of variables. 

For the Peace Corps trainees these predicted peer-ratings correlate 
higher with the actual peer ratings in every case than do the 
corresponding self-ratings. In addition, the predicted peer-ratings 
and the self-ratings are themselves highly correlated in eacn cas , 
ranging from .69 for Factor II to .88 for Factor I. While these 
latLr findings are surely partly a function of a common item and 
rLponse format and a cormnon source (the individual respondents) it 
should be borne in mind that they ai^e, in each case, 
on only a single response per subject per scale. These data 
accordingly less reliable than are the peer-rating measures which 
are based on averages over multiple respondents on each scale. 

Thus it would seem that these respondents expect their peers 

to rate them slightly differently than they rate 
this expectation they are correct in a manner and direction that 
reflect a degree of perceptual accuracy. However, they expect their 
peers* ratings of them to conform much more closely to their own 
self-ratings than does in fact occur. 



Note Factor score coefficients derived from independent sample of 169 fraternity males. 

Decimals omitted. 
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Validity Generalization of the Factor Score Coefficients 

Sample of 56 Fraternity Seniors 



Note — Factor score coefficients derived from an independent sample of 169 Fraternity males. 
Decimals omitted. 
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The results from tite second sample where the subjects were 
not previously acquainted with one another are, not surprisingly, 
much more mixed and irregular in their pattern. Not only are the 
relationships between peer-ratings and self-ratings lower for these 
subjects than has been previously observed for samplesof close 
associates, but their predicted peer ratings are also less highly 
and less uniformly related to both the actual peer-ratings and self- 
ratings as well. These subjects knew they were being rated by 
virtual strangers and the correlations of their predicted peer- 
ratings with the other sets of measures no doubt reflect this fact. 

As you are all aware, the title of this presentation was taken 
from Robert Bums* ode, "To a Louse" and is the second line of the 

couplet : 



0 wad some Pow*r the gif tie gie us 
To see oursels as others see us! 



Whether our subjects yearned so deeply as the illustrious Scot for 
such a gift seems doubtful; our results indicate that at least 
not all of them were richly blessed. But it would appear that they 
were not wholly unaware either of how others saw them and, indee , 
they agreed to a moderate extent with these external judgments. 

On the other hand, they clearly projected onto their peers an unwarranted 
amount of agreement with their own self appraisals. Perhaps this 
is what Bums* lament was all about. 



Reprise and Outlook 



At the outset of this research program we were willing to consider 
the collective perceptions of others as reflected in the peer- 
ratings as a set of criteria against which self-report measures 
could be validated. But a little thought and several sets of such 
data (e.g., Norman and Harshbarger, 1965; and Passini and Noman, 

1966) revealed a number of methodological and substantive deficiencies 
that exist in these measurements, also. Constraints owing to 
forced nomination fomats and procedures, interrater disagreements 
based on differential or limited exposure to the ratees, and 
ambiguities of the trait descriptors all serve to attentuate the 
amount of construct valid variance available in the peer ^^atings. 

When one considers in addition the potential distortions which accrue 
to commonly used analysis methods that, however conventional, 
are more or less ill-suited for these sorts of data, it is ^a^dly 
surprising that convergent relationships rise no higher than those 

thus far obtained. 

I don*t wish to be recorded as an apologist for these and 
previous sets of findings. But I do think that we have been making 
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Table 11 

Intercorrelations of Factor~Score Estimates Derived from Peer 
Ratings, Self-Ratings, and Predicted Peer Ratinss for Two of the Samples 
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^For il? * 73, 2> 1.195 is significant at p <.05 v -tailed) and r >_ |±.232| 
Is significant at p <.05 (2-tailed) . 

'^For « 84, 1 .181 is significant at p <.05 (1-tailed) and r 1 l±.215l 

is significant at p <.05 (2-tailed). 

Decimals omitted. 




progress in recent years on the development of personality aseess- 
ment instruments and methods, on identifying and estimauing the 
magnitudes of (and relations among) various methodological and 
substantive components of the score variables with which we 
work, and on bringing a more reasonable and sophisticated attitude 
to bear on the analysis, interpretation and use of these kinds 
of data for theory development and applied purposes. But we have 
neither exhausted the possibilities inherent within our present 
approaches nor have we fully exercized the ingenuity and per- 
severance needed to invent new and better methods for assessment 
and for analyzing these sorts of data. 

In this latter regard it seems to me that the day has passed 
when anything of much value is going to be learned by "one-shot" 
studies using just any conveniently available inventory, sample 
and data analysis procedure. The need at the present is for 
systematic investigations that are extensive in scope, that are 
both multivariate and multi-method in design, and that span an 
appreciable range of variation on developmental, demographic, 
and situational facets. 

It is unlikely also that "canned" analysis programs are going 
to suffice for processing the kinds of data that these more 
elaborate studies will yield. Such methods were, by and large, 
designed for other, and simpler, models and they often serve 
effectively for such uses. But their suitability for sorting 
out the diverse kinds of information implicit in complex batteries 
administered to stratified samples under multiple task conditions 
is, in many cases, highly questionable. 

The general point I am trying to make in closing is that 
there is nothing simple, routine, or automatic about research 
on personality assessment these days; either in the design of 
investigations or in the analysis and interpretation of results. 

If we are to make the progress that now seems possible in this 
area we will have to have the courage and perseverance to under- 
take large and complex investigations on a programmatic basis 
and the ingenuity to adapt or invent methods of analysis to the 
demands that such research designs present. 
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